Motion information is a very important aspect of image generation. For example, in many frame sequences, the only difference between one frame and the next is the relative motion of objects within the frames. Video compression takes advantage of that fact by encoding image data, for example, every fourth frame. For the frames in between, only the information needed to generate the missing frames is stored. Part of the information needed to generate the missing frames is information about the motion of the objects within the frames.
Another example of where motion information is important occurs when a frame rate of an input signal is inconsistent with the frame rate needed for an output device. Typical movie films are recorded at 24 Hz, 25 Hz, or 30 Hz. Picture rates of common video cameras are 50 Hz and 60 Hz. Commercially available television displays, on the other hand, have picture rates up to and beyond 120 Hz. To interface broadcast video with a high-end TV display, the original sequence from the broadcast video needs to be up-converted using, for example, a picture rate converter. A picture rate converter typically operates by interpolating image frames at time instances where the frame sequence from a lower-frequency source device has not yet been converted for a higher-frequency destination display.
In simple picture rate converters, a picture is often repeated in the destination display until the next picture arrives from the source device, which oftentimes results in blur and judder when motion occurs. Motion estimation and compensation circuits may be used in a picture rate converter to reduce these unwanted effects and achieve a high performance conversion for moving sequences. Motion compensation operates by estimating where objects of an interpolated picture would be, based on the direction and speed of the movement of those objects. The direction and speed values may then be expressed as motion vectors and are used to “move” the objects to the correct position in a newly generated frame. If this technique is applied correctly, its impact may be immediately visible on any picture sequence involving motion, where the resulting pictures can hardly be distinguished from the original sequences before the up-conversion.
FIGS. 1A and 1B illustrate two frames (frames A and B, respectively) that show a typical scene with periodic structure and the artifact (seen in FIG. 1A) caused due to the periodic structure. There is a direct correlation between the motion between the frames and the pitch (period) of the periodic structure. Let us assume that the motion of the pixels including the periodic structures between frame A and frame B is 10 pixels and the pitch (period) of the periodic structure is 12 pixels. Thus, when we find the correlation between frame A and B, there is a dual relation between the motion and the pitch. The correlation surface resulting from application of either transform domain techniques or spatial domain techniques will show a match at the pixel located 10 pixels from the origin as well as at the pixel located −2 pixels from the origin. If we generalize the case, then we will see that if the pitch is (M, N) and the motion is (X, Y) in the vertical and horizontal directions, respectively, then due to the duality, we will have correct motion information of X and Y as well as incorrect motion information of (X−M) and (Y−N). When the motion is assigned to the dense pixel field, the incorrect motion information of (X−M) and (Y−N) will cause objectionable artifacts. Thus, one of the major artifacts present in motion compensated images is motion vector errors due to the presence of periodic structures. Most of the motion compensated algorithms show this artifact as it is a difficult problem to solve without relevant image specific information.